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Abstract 

In this paper, we have established a new framework of truncated inverse sampling for 
estimating mean values of non-negative random variables such as binomial, Poisson, hyper- 
geometrical, and bounded variables. We have derived explicit formulas and computational 
methods for designing sampling schemes to ensure prescribed levels of precision and confidence 
for point estimators. Moreover, we have developed interval estimation methods. 

1 Introduction 

Parametric estimation based on sampling is an important branch of mathematical statistics with 
ubiquitous applications across many fields, from operation research, biology and medical science, 
agriculture science, computer science, social science, telecommunication engineering, control en- 
gineering, to name a few. A wide class estimation problems of both theoretical and practical 
significance can be put into the setting of estimating the mean value of a random variable via 
sampling. Familiar examples include the estimation of binomial parameters, Poisson parameters, 
finite population proportion, the mean of a bounded variable, and so on. A simple yet frequently 
used sampling scheme for estimating the mean value a random variable X is to draw samples of 
X until the sample sum is no less than a prescribed threshold and then take the empirical mean 
as an estimate for the true mean value. This sampling scheme, referred to as inverse sampling, 
was first studied by Haldane [HI [9] in the context of estimating a binomial parameter. Recently, 
inverse sampling has been studied by Chen [Il[2], Dagum et al. [5] and Cheng [3] for estimation 
of the mean of a bounded variable. Mendo and Hernando [lUj have revisited inverse sampling for 
estimating binomial parameters. 

Theoretically, there is no limit on the number of samples for inverse sampling. However, the 
practical situation is quite contrary. Due to the limitation of resources, almost every practitioner 
would specify a maximum sample size on the sampling. This means that the frequently used 



*The author had been previously working with Louisiana State University at Baton Rouge, LA 70803, USA, 
and is now with Department of Electrical Engineering, Southern University and A&M CoUege, Baton Rouge, LA 
70813, USA; Email: chenxinjia@gmail.com 



1 



method is actually the truncated inverse sampling scheme in the sense that sampling is continued 
until the sample sum is no less than a prescribed threshold or the number of samples reach the 
maximum sample size. 

While the ideal inverse sampling has drawn extensive research effort, little attention has been 
paid to the theoretical issues of the truly useful truncated inverse sampling scheme. In this paper, 
we shall investigate the essential theory of truncated inverse sampling with a prevailing theme of 
error control. We have answered two equally central problems regarding pre-experimental plan- 
ning and post-experimental analysis. The first problem is on the determination of the threshold 
value and the maximum sample size for guaranteeing prescribed levels of precision and confidence 
of an estimator. The second problem is on interval estimation of the parameter based on the 
observed data when the truncated inverse sampling is completed. 

The remainder of the paper is organized as follows. In Section 2, we present our general results 
for truncated inverse sampling. In Section 3, we consider the problem of estimating binomial 
parameters. In Section 4, we discuss the estimation of the proportion of a finite proportion. In 
Section 5, we discuss the estimation of Poisson parameters. The estimation of the mean of a 
bounded variable is investigated in Section 6. Section 7 is the conclusion. 

Throughout this paper, we shall use the following notations. The expectation of a random 
variable is denoted by E[.]. The set of positive integers is denoted by N. The ceiling function and 
floor function are denoted respectively by [.] and [.J (i.e., [x] represents the smallest integer no 
less than x; [xj represents the largest integer no greater than x). The gamma function is denoted 
by r(.). For any integer m, the combinatoric function with respect to integer z takes value 



associated random samples Xi,X2, ■ ■ ■ are parameterized by 6. The parameter 9 in Pr{. | 6} may 
be dropped whenever this can be done without introducing confusion. The other notations will 
be made clear as we proceed. 

2 General Theory 

In this section, we shall develop some general results on the truncated inverse sampling. Let X 
be a non-negative random variable defined in a probability space {Q, ^ ,Fr). Our problem is to 
estimate the mean, = E[X], of X based on i.i.d. random samples Xi,X2, • • • of X. To this end, 
we shall adopt a truncated inverse sampling scheme as follows: 

Continue sampling until the sample sum is no less than a threshold value 7 > or the number 
of samples reaches an integer n. 

Let n be the total number of samples when the sampling is stopped. By the definition of the 
truncated inverse sampling scheme, n is a random variable such that 



r(m+l) 




r(z+i)r(m-2+i) 
The notation 
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for any to G 0,. Define k = Then, we can take /i = as the estimator for 

H = E[X]. 

With regard to the distribution of /i, we have 
Theorem 1 For any z > 0, 

^S-^ X /Pr{Ell('^''^. <7} for^Knz, ^ ^ , /Pr{Ell('^ > 7} for^<nz, 
Pr|/i < z\ ~ < Prj/^ > z\ — < 

I {X^iLi — J > nz I Pr{E"^j^ > riz} for 7 > nz. 

With regard to the average sample number E[n] of the truncated inverse samphng associated 
with random variable X, we have 

Theorem 2 For any non-negative random variable X with positive mean and finite variance, 
E[n] < min{n, ^ + 1}. Specially, if ^ is a positive integer and X is a Bernoulli random variable 
such that Pr{X = 1} = 1 - Vt{X = 0} = p G (0, 1), then E[n] < niin{n, ^}. 

3 Estimation of Binomial Parameters 

In this section, we shall consider the estimation of a binomial parameter based on truncated 
inverse sampling. Let X be a Bernoulli random variable such that Pr{X = 1} = 1 — Pr{X = 
0} = p € (0, 1). Our goal is to estimate p based on i.i.d. random samples Xi, X2, • • • of X. Since 
Xi assumes only two possible values or 1, the threshold value 7 shall be restricted to an integer. 
The estimator for p can be taken as p = = ^, where k and n have been defined in Section 

2. 

In order to estimate p via truncated inverse sampling, a critical problem is the determination 
of the threshold value 7 and the maximum sample size n. By making use of functions 

J^b{z, /i) = z In z) In J^t) ' -^i^z, /u) = ^^b{z, 

for < z < 1 and < ;U < 1, we have derived the following result. 

Theorem 3 Let < 6 < 1. Let < £a < £r < ^ be respectively the margins of absolute and 
relative errors such that + Ea < ^- Then, Pr||p — p| < Sa or < > 1 - 5 provided that 

n > yy M and 7 > where p* = 

Theorem 3 provides explicit formulas for determining the threshold value 7 and the maximum 
sample size n. To reduce conservatism, we can take a computational approach to obtain smaller 
7 and n. In this direction, the following theorem which is of fundamental importance. 

Theorem 4 Let ^{■) and ^{.) be monotone functions. Let the supports of J^ij)) and ^(p) be 
denoted by and F^ respectively. Then, the maximum of Pr{p < ^(p) | p} with respect to 
p € [o, b] C [0, 1] is achieved at n [a, b] U {a, 6} provided that has no closure point in [a, 6]. 
Similarly, the maximum of Pr{p > (p) \ p} with respect to p ^ [a, b] C [0, 1] is achieved at 
la^ n [a, 6] U {o, 6} provided that has no closure point in [a, b] . 
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In Theorem 4, we have used the concept of support. The support of a random variable is 
referred to the set of all possible values that the random variable can assume. By virtue of 
Theorem 4, we have obtained the following results. 



Theorem 5 Let < 6 < 1 and C > 0. Let < Ea < £r < ^ i>e respectively the margins of absolute 



and relative errors. Define n ~ 



^B(P*+ea,P*) 



and 7 = 



4'i(p*+ea,P*) 



with p* = ^. Define as 



the support ofp — Ea, support ofp + Ea, St support ofp/{l + Er), as the 

support ofp/ (1 — Er). Then, Pr ||p — p| < Sa or 2z£ < > i — ,5 provided that 



Pr{p >p + Ea\p}<-, Vp G ^~ U {p*} n (0,/] 

Pt{p <p-Ea\p}<-, Vp G ^+ u {p*} n (0,/] 
Pt{p > p{i + Er) b} < 2 ' G ^+ n {p\ 1) 

Pt{p < p{l - Er) b} < 2' n {p\ 1) 



(1) 

(2) 
(3) 
(4) 



where these conditions are satisfied when C, is smaller than h. 



Clearly, the support of p is : j = 0, 1, • • • , 7} U : m = 7, 7 + 1, • • • , n — 1}. Theorem 
5 asserts that the prescribed levels of precision and confidence can be guaranteed if C is small 
enough. Hence, we can determine an appropriate value of by a bisection search method. 

When the sampling is terminated, it is desirable to construct a confidence interval for p. For 
this purpose, we have 

Theorem 6 Let < S < 1. Define lower confidence limit p G [0, 1) such that p = for k = 
and that X]"=k ~ P)"^' = f for k > 0. Define upper confidence limit p G (0, 1] such that 

p=l fork = n and that J2^^^ - p)""' = | /or k < n. Then, Pr{p <p<p}>l-5. 



It should be noted the approach of constructing a confidence interval for p can be considered 
as a generalization of Clopper and Pearson's method |4j of interval estimation. 



4 Estimation of Finite Population Proportion 

In the last section, we have investigated the estimation of a binomial parameter p, which can be 
considered as the proportion of an infinite population. In many situations, the population size is 
finite and we shall devote this section to the estimation of the proportion of a finite population. 
Consider a population of units, among which there are M units having a certain attribute. It is a 
frequent problem to estimate the population proportion p = ^ hy sampling without replacement. 
The procedure of sampling without replacement can be precisely described as follows: 
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Each time a single unit is drawn without replacement from the remaining population so that 
every unit of the remaining population has equal chance of being selected. 

Such a sampling process can be exactly characterized by random variables Xi, ■ ■ ■ , Xjv defined 
in a probability space (fi, Pr) such that Xi denotes the characteristics of the i-th sample in 
the sense that = 1 if the i-th sample has the attribute and Xi = otherwise. By the nature 
of the sampling procedure, it can be shown that 



Fr{Xi = Xi, i = 1, • • • ,n} 



M \ / N -M 
E"=i XiJ \n - X^^^i Xi 



n \fN 



(5) 



for any n S {1, ■ ■ • , N} and any Xi €{0,1}, i = 1, ■ ■ ■ , n. Moreover, if the proportion p = ^ is 
fixed and the population size tends to infinity, the sequence Xi, X2, • ■ ■ , X^r tends to the i.i.d. 
random samples of a Bernoulli variable. 

To estimate the population proportion p, we can use a sampling scheme defined by positive 
integers 7 and n as follows: 

Continue sampling without replacement until 7 units found to have a certain attribute or the 
number of samples reaches n. 

Despite the lack of independence in the sequence Xi, X2, • • • , Xn with joint distribution ([5]), 
such a sampling method is also referred to as truncated inverse sampling due to the fact that, 
when the sampling is terminated, the number of units having a certain attribute, denoted by k, 
is actually equal to EiLi where n is the sample size when the sampling is terminated. This 
implies that, by relaxing the independency assumption, we can put such a sampling scheme in 
the general framework of truncated inverse sampling described in Section 2. It can be seen that, 
as the sample size tends to infinity while the proportion p is being fixed, such a sampling scheme 
reduces to the truncated inverse sampling for the estimation of a binomial parameter as discussed 
in Section 3. 

As in the case of estimating a binomial parameter in Section 3, the estimator for the proportion 
of a finite population can be taken as p = """^'^'"'^-^ = ^ . In order to determine n and 7 to guarantee 
prescribed levels of precision and confidence, we have the following result. 

Theorem 7 Let < 5 < 1. Let < < < 1 respectively the margins of absolute and 
relative errors such that + < | ■ Then, Pr {|p - p| < Sa or |p — p| < pSr} > I — 5 provided that 
n > yy M and 7 > ^^4^^^, where p* = 

Theorem 7 provides explicit formulas for determining threshold value 7 and the maximum 
sample size n. To reduce conservatism, we can take a computational approach to obtain smaller 
7 and n. In this direction, the following theorem is useful. 

Theorem 8 Let ^{.) and ^(.) he non- decreasing integer-valued functions. Let the supports 
of .^{p) and ^(p) he denoted hy and L^/ respectively. Then, the maximum 0/ Pr{Af < 
J^{p) I M} with respect to M ^ [a, 6] C [0, A^], where a and h are integers, is achieved at 
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/_5f n [a,b] U {a,b}. Similarly, the maximum o/ Pr{M > ^{p) \ M} with respect to M ^ [a, is 
achieved at lo^/ n [a, U {a, 6}. 

By virtue of Theorem 8, we have obtained the following results. 



Theorem 9 Let < 5 < 1 and ^ > 0. Let < < £r < 1 respectively the margins of absolute 



and relative errors. Define n 



and 7 



^^th p* = f^. Define as 



the support of [N{p — ea)\, "-^ support of \N{p+£a)], -^t support of \Np/ {l+er)\, 

as the support of \Np/ (1 — e^)] . Then, Pr {\p - p\ < £a or |p — p| < per] > I — 5 provided that 



Pr{p >p + £a\M}<-, VM G U { [iVp*J } n (0, Np*] (6) 

Pr{p < p - ea I M} < -, VM G ^+ U { [Np*\ } n (0, iVp''] (7) 

Pr{p > p(l + £r) I M} < -, VM e ^+ U { [iVp*J + 1} n (iVp^ A^) (8) 

Pr{p < -£r)\M]<-, VM G U { [iVp*J + 1} n {Np\ N) (9) 



where these conditions are satisfied when C, is smaller than i. 



Clearly, the support of p is : j = 0, 1, • • • , 7} U : m = 7, 7 + 1, • • • , n — 1}. It is asserted 
by Theorem 9 that the prescribed levels of precision and confidence can be guaranteed if C, is small 
enough. Therefore, an appropriate value of Q can be determined by a bisection search method. 

In order to construct a confidence interval for M, we have 

Theorem 10 Let Mi be the smallest integer such that Er=k (^0 (^n-^0/(n) > f- be 
the largest integer such that ^^o (^") (^„^")/(n) > l- ^^^n, Vt{Mi <M< M„} >1-5. 

With regard to the average sample number E[n], we have 

Theorem 11 // the population proportion p is positive, then E[n] < min|?i, 

5 Estimation of Poisson Parameters 

Let X be a Poisson random variable with mean A > 0. It is a frequent problem to estimate A 
based on i.i.d. random samples Xi, X2, • • • of X. This can be accomplished by using the truncated 
inverse sampling scheme described in Section 2. Since Xi is an integer-valued random variable, 
we shall restrict the threshold 7 to be a positive integer. We take A = """^'^'"''-^ as the estimator 
for A, where k and n have been defined in Section 2. 

To determine the threshold 7 and the maximum sample size n, we need to have an upper 
bound for A. We do not pursue results along this line. We are more interested in the construction 
of a confidence interval when the sampling is completed. For this purpose, we have 
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Theorem 12 Let < 6 < 1. Define lower confidence limit A such that A = for A = 0, 
T,Z^U^Xyexp{-nX) = lforX>^, and ES_k Ii (^A)^ exp(-nA) = | /or < A < ^, 
where k = Yli^=i -^i- Define upper confidence limit A such that A = cxd for n = 1, J2]^q [(n — 
l)A]' exp(-(n - 1)A) = | /or ^ < A < 7, and Eto IlM)' exp(-nA) = | /or A < ^. Then, 
Pr{A < A < A} > 1 - (5. 



It should be noted that the interval estimation method described in Theorem 12 is a general- 
ization of Garwood's interval estimation method [6]. 



6 Estimation of Bounded- Variable Means 

Let X be a random variable bounded in [0,1] with mean fi = E[X]. In many situations, it is 
desirable to estimate fi based on i.i.d. random samples Xi,X2, • ■ ■ of X (see, e.g., [5] and the 
references therein) . To fulfill this goal, we shall make use of the truncated inverse sampling scheme 
described in Section 2. In order to determine the threshold 7 and maximum sample size n to 
guarantee prescribed levels of precision and confidence, we have 

Theorem 13 Let < 6 < 1. Let < < < 1 &e respectively the margins of absolute and 
relative errors such that p* + £a < ^ with = Then, Pr — < Sa or < e,.| > I — S 

provided that 

l-Er In I In I Inf 
7> > 7> / , / ^ T' 7 > y,^^ TT, n> 



With regard to the interval estimation oi fj,, we have 
Theorem 14 Let < 6 < 1. Define lower confidence limit /x G [0, p.] such that /x = for p. = 0, 

^ In — ^ ^ In — ^ 

./#b(a^)/^) = /o?" < < ^, and ^i{'jl,fi) = for /x > ^. Define upper confidence limit 

^ ^ In— ^ /^\ln- 

JZ e [fi, 1] and that JI = 1 for p, > .^b{P,V) = -f^ for p < ^, J^i y^^.'Ji) = -f- and 

^<ji<lfor^>ri>l. Then, Pr{^ < < 71} > 1 - ,5. 



7 Conclusion 

In this paper, we have established a general theory of truncated inverse sampling for estimating 
the mean value of a large class of random variables. We have applied such a theory to the 
common important variables such as binomial, Poisson, hyper-geometrical, and bounded variables. 
Rigorous methods have been derived for determining the thresholds and maximum sample sizes 
to ensure statistical accuracy. Interval estimation methods have also been developed. 
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A Proof of Theorem 1 



The theorem can be shown by estabhshing Lemmas 1 to 4 as fohows. 

Lemma 1 Suppose z> ^. Then, Pr{/i < z} = P^^lYl^i -^i < 7} where m = — 1. 

Proof. By the assumption that z > ^ and the definition of the samphng scheme, 

iu> z, k<7}=|— >z, k<7lc |— >-, k<7l = j— >-, k<7, n = n 
[n J In n J n 

Therefore, {Jl > z} = {ft > z, k < 7} U {/i > z, k > 7} = {/i > z, k > 7} = > z, k > 7}. 
To show the lemma, it remains to show that |^ > z, k > 7} = Pr{J2^i ^ 7}- Since ah Xi are 
non-negative, we have > z, k > 7} {n < m, k > 7} C {n < m, Y."Li Xi>j}<Z {Y,"Li > l}- 
On the other hand, by the assumption that z > ^, we have m = |"^] — 1 < n — 1. Hence, by the 
definition of the samphng scheme, we have {X]j=i -''^^ > 7} ^ {n < m, k > 7, J2^Li > 7} ^ {n < 
m, k > 7}. It fohows that {J2Zi^^ > 7} = {n < m, k > 7} = > z, k > 7} = {/i > z}. This 
completes the proof of the lemma. 

□ 

— ' — ^f- 

Proof. By the assumption that z < ^ and the definition of the sampling scheme, 

{/i < z, k > 7} = f- < z, k > 7I C < ^, k > 7I = {n > n, k > 7} = 0. 
L n Jinn ) 

Therefore, 

{/i < z} = {/i < z, k < 7} U {/i < z, k > 7} = {/i < z, k < 7} = 1^ < z, k < 7I 

= {^<.,k<7,n = n}.|^^<z,f:X.<7,n = n|c|^^k^<z}. 

On the other hand, by the definition of the sampling scheme and the assumption that 2; < ^, 
we have {^%^ < z} C {^^^ < z, n = n} = {^%^ < z, Y2=i < 7, n = n} = {/i < z}. It 
follows that {/i < z} = | ^'=^^ ^' < z|. This completes the proof of the lemma. □ 

Lemma 3 Suppose z > ^. Then, Pr{/x > z} = Pr{^^]^ Xi > 7} where m = |_^J . 
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Proof. By the assumption that z and the definition of the sampling scheme, 



{/I > z, k < 7} = 1^ > 2, k < 7I = 1^ > z, k < 7, n = 7i| = 0. 

Therefore, {fi > z} ^ > z, k < 7} U {/i > z, k > 7} = {/i > z, k > 7} = > z, k > 7}. To 
show the lemma, it remains to show that > z, k > 7} = Pr{X]i!Li — 7}- Since all Xi are 
non-negative, we have > z, k > 7} = {n < m, k > 7} C {n < m, YlT=i ^ 7} ^ {J2iLi > 7}- 
On the other hand, by the assumption that z > ^, we have m = |_^J < n. By the definition of the 
sampling scheme and the fact that all Xi are non-negative, {X^Hi Xi > ^} (Z {n < m, X)™ 1 ^ 
7} ^ {n < m, k > 7}. Hence, {J27Li Xi > ^} = {n < m, k > 7} = > z, k > 7} = {/i > z}. This 
completes the proof of the lemma. 

□ 

Lemma 4 Suppose z < ^. Then, Pr{/x > z} = Pr | ^'"^ — -hi- 
proof. By the assumption that z < ^ and the definition of the sampling scheme, 

{/i < k > 7} = f- < ^, k > 7I C (1 < ^, k > 7I = {n > n, k > 7} = 0. 
In Jinn J 

Therefore, 

{/X < z} = {/I < z, k < 7} U {/I < z, k > 7} = {/I < z, k < 7} = j - < z, k < 7 



n 



1^ < z, k < 7, n = ?i| = I ^'-^ < z, < 7, n = 7i| C I 



< z 



On the other hand, by the definition of the sampling scheme and the assumption that z < ^, 
^^^^<z|c| ^^=i^' <z, n = n| = <z, E^' <7, n = n| = {/i < z}. 

It follows that {fi < z} = |^%^ < 2}, i.e., {fi > z} = |^%^ > -zj- This completes the 
proof of the lemma. 

□ 



B Proof of Theorem 2 

By the definition of the truncated inverse sampling scheme, 

{ n \ n f m — 1 m ^ 
<7U J^mPri ^Xi<7, ^Xi>^\ 

1=1 J m=l I i=l i=l ) 

J^Xi < 7 I + ^ nPr i X, < 7, > 7 i = n. 

i=l J m=l I i=l 2=1 J 



By the fact that Xi is non-negative, 



\m—n+l \ i—1 



4=1 



. j=l 



Since E[X] = /i is positive and the corresponding variance a"^ is finite, we have, by Chebyshev's 
inequahty. 



< lim Pr. 

k—^oo 



> 



7 



< lim 



Hence, 



E[n] = ^ nPri ^X,<7, ^X,>7 +E'"PM E^'<^' E^»^ 



m— n+1 



' ?7l — 1 



2^1 



< E"*pmE^*<^'E^«^^P^[™]' 

where m is the sample number of the classical inverse sampling scheme with the following 
stopping rule: Sampling is continued until the sample sum is no less than 7. By the definition 
of the classical inverse sampling, we have Xli^T^ ^« < 7- ^PPlyiiig Wald's equation, we have 
EE^Y^ Xi] = E[m - 1] E[X] < 7, which imphes that E[m] < ipq + 1 = ^ + 1- Since E[n] is less 
than both n and E[m] as shown above, we have E[n] < min{n, ^ + 1}. 

In the special case that 7 is a positive integer and that X is a Bernoulli random variable such 
that E[X] = p G (0, 1), we have Yl'iLi = 1 a-iid consequently, by Wald's equation, E[^™ Xi] = 
E[m] K[X] = 7, from which we get E[m] = = ^ and it follows that E[n] < min{n, ^}. This 
completes the proof the theorem. 



C Proof of Theorem 3 

We need some preliminary results. The following lemma is a slight modification of Hoeffding [TJ. 

Lemma 5 Let Xi, ■ ■ ■ ,X„ be i.i.d. random variables bounded in [0, 1] with common mean value 
Id e (0, 1). Then, Pr| ^'=;^' > z| < cxp {n^B{z, ^)) forl>z>fi. Similarly, Pr < z| < 

exp (n^^iz, fi)) for < z < fi. 

Proof. For z = fi, we have Pr | ^' > < exp (n^B(z, m)) = 1- For fi < z < 1, it was shown 
by Hoeffding in [7j that Pr > < exp (n..#B(z, m))- For z = 1, we have Pr > z| = 

nr=i Mx^ = 1} < nr=i nx^] = = exp (n.^B(i, m))- 

For z = 0, we have Pr{£%^ < z} = nLi(l " ^ 0}) < nLi(l - nx^]) = (1 - m)" = 

exp (n^B(0, ^)). For < z < /i, it was shown by Hoeffding in [7] that Pr | ^'^^ < z| < 
exp (n^B(2, m))- For z = fi, we have Pr | ^'^^ < zl < exp (n^B(z, ^)) = 1- ^ 
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Lemma 6 Let < e < 1. Then, + e/-f,/i) is monotonically decreasing with respect to 

/U G ( 0, J . Similarly, — efi, fx) is monotonically decreasing with respect to ^ ^ (0, 1) . 



Proof. Note that ^-^'^^+"^'^^ 



In 



In (1 - 



< - 



< if In 



1 — yU / 1 — M 



(10) 



As a consequence of < /i < jq:^, we have < < 1. Since ln(l — x) < —x for any x E (0, 1), 
it fohows that (|10p holds and thus + is monotonically decreasing with respect to 

/^e(o,^). 



Similarly, to show that — e^, fx) is monotonical 

< if In 



0/1 



^2(1-6) 



In 



l-M(l-e) 



y decreasing with respect to /x, note that 



1-^ 



ha(l + ^l< 



> i e 



1 — jU / 1 — Ai 



(11) 



Since > and ln(l+a;) < x for any x E (0, oo), we have that (jlip holds and thus ^j{^ — £fi, fi) 
is monotonically decreasing with respect to /x E (0, 1). 

□ 



Lemma 7 + efi, ji) > — efi, ji) for /i E (O, ^) and < e < 1. 



Proof. Direct computation shows that 



de 



(1 + £)V 



In 



l-(l+£)/i 



ifi~efi, At) 



1 



In 



1-M 



Since In 



< l-(l+e)M a'^^ 1^ 



d^i{fj, + e/i, /i) d^i{^ — e/i, /i) 



as 



9£ 



> 



1 



de (1 — e)^/i 

< - i-(i-e)p ' we have 

e/.t 1 



(l + £)2Ml-(l+£)Ai (l-e)2/il-(l-e)^ 



l-(l-e)M 



> 



if (1 + ef[l - (1 + e)^x] - (1 - e)2[l - (1 - £)^A > 0, or equivalently, Ae - 2e(3 + e^);^ > o, which 
is true because Ae — 2^(3 + e^)/i > 4e — 2e(3 + 1) x i = as a result of < e < 1 and < < ^. 
The lemma immediately follows from the fact that ^'^'^g^"^^'^^ is greater than '^'^'^g^'^^'^^ for 
0<e<l, Q < ji <\ and + e^, ji) is equal to — e/i, /x) for e = 0. 

□ 



Lemma 8 Lei < e < ^. Then, ^i{pL + e,/i) is monotonically increasing with respect to 
/t E (O, ^ — e) . Similarly, — e, /i) is monotonically increasing with respect to ^ ^ (e, ^ + . 
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Proof. It can be shown that ^^^^ = In (^) + ^t.^^,,,,, > if In (^) < 



£(H±£^. Since In [j^) < it suffices to have ^ < or equivalently, ^(1 - fi) < 

(1 — /i — e)(/z + e) = — /^) — e/i + (1 — ;u)e — e^, which can be ensured by < < ^ — e. This 
proves the first statement of the lemma. 

Similarly, -^'(^"'''^ = -r-^rlnfT^l - ^ — hr-^ > if In f y^^) < -4F4- Since 
1^ (t^TI+i) < "T:^' to ensure '^'^^"''■'"^ > 0, it suffices to have -j:^ < or equivalently, 

fi{l—fi) > (1— ^+e)(/i— e) = which can be guaranteed by e < fi < ^+f- 

This proves the second statement of the lemma. 



□ 



Lemma 9 ^i{fi + e,fi) > — e, /i) for < e < ^ < \. 

Proof. It can be verified that = In (t^) and ^ ^ In 



Since In ( j^^] < and In ( ) < to ensure ^^■^^+"-^^ > it suffices 

to have — (^^^^2 > — (^:^g)2 i-^+g i or equivalently, 2// — > e^, which is true because 

2/i - - > 2^ - 3;u2 - = 2/i(l - 2;U.) > as a result ofO<e</x<i. Therefore, 
the lemma is true since + e,/i) = - e,/i) for e = and ^^''^+"'^^ > ^■^■(^-^■a') for 

< e < ^ < i. This completes the proof of the lemma. 

□ 



Lemma 10 Let < e < |. Then, ^b(a* + is monotonically increasing with respect to 

/U € (O, ^ — e). Similarly, ^b{I-I' — ^il^) is monotonically increasing with respect to jj G (e, 

Proof. Our computation shows that 



dfj. (/i + e)(l-/i) ^(1 - /i) 9ju9e - ^) (/i + e)(l - ^ - e) 

Since ^-^B(M+g,A') ^ for e = and ^"'^g^^^^'^^ > for e < i - ^u, it must be true that 
^•'^B{^+£,fj') y g £qj, ^ ^ ^0, ^ — e). This proves the first statement of the lemma. Similarly, we 

can show that ^-^bIm-s.^) ^ M(Wi±£) _ 3^.^B(M-e,A.) ^ ^ i _ gince 

^^B(p-e,^^) = for e = and ^'^g/^"^'^^ > for < e < < i it must be true that 



(9/1 

'^■'^Bi^-£,iJ') y Q £qj, < e < /i < ^. This proves the second statement of the lemma. 



□ 



Lemma 11 LetO<e<^. T/ien, ^b(^ + ^) > '^B(/i — Z^) /or /-i G (e, |) . 
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Proof. Straightforward computation shows that 



In 



/i 1 — — e 
1 — ^ /X + e 



de 



In 



H 1 — fi + e 
1 — /i n — e 



In- 



n2_ 2 _ -J 

i_ -i — >Oife<^< 2. By virtue of such result and 



inUS, g-^ gj - m ^^_^yi / ^ O ^ ^ 2 ' 

the fact that ^b(/x + = ^b(/x — for e = 0, we have ^b(/x + ^) > -^b(/J — for 
e < < ^. This proves the lemma. 

□ 



Lemma 12 Lei < e < 1. Then, ^Bit-i- + monotonically decreasing with respect to 

G iO, i^j- Similarly, ^b(^ — is monotonically decreasing with respect to fi £ (0, 1). 



Proof. The first statement of the lemma is true because 



= (!+£) In 



1 - 



< (1 + £) X 



(l + £)(l-/i) 



(l + £)(l-/i)J ■ 1-M 
for < ;U < Similarly, the second statement of the lemma is true because 



= 



for < < 1. 



= (1 -e)ln 



1 



{l-e)il-^I) 



< (1 -e) X 



1-M 



= 



□ 



Lemma 13 Let < e < 1. Then, 



e/i, /i) for /i G (O, i) . 



Proof. It can be shown by tedious computation that 



(9-^b(m + m) , 1 — u — eu, 
— = /U In ■ 



Hence, 



de 



(l + 6)(l-^)' 



;(^ - £/i,/i) 



In ■ 



1 — fi + efi 



de 



^ In 



1 — fj, — efi 1 — 11 + en 



> 1 for < /i < i , we have 



(9e Se 



omce - (i„p)2_e2(i_^)2 / ^xw. ^ ^ 2 

for < /i < ^. Noting that ^b(/x + £^5/^) — ^b(a* ~ ^/^lA*) = for e = 0, we have ^b(/x + 
e/i, /i) — ^b(/J — ^A^) a*) > for < /i < ^. This completes the proof of the lemma. 

□ 



Lemma 14 Pr{p > (1 + Er)?*} < | for any p € (p*, 1). 
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Proof. To prove the lemma, we shall consider the following three cases: 
Case (i): (1 + er)p > 1; 
Case (ii): ^ < (l + e.)p< 1; 
Case (iii): (1 + er)p < ^. 

For Case (i), it is obvious that Pr{p > (1 + Sr)p} = < |. 
For Case (ii), applying Theorem 1 with z = {1 + er)p > ^, we have 

Pi{p > (1 + er)p} = Pr I ^ > 7 

< exp(^L7AJ (p^'^'l ) (12) 

< exp(7^i(z,p)) (13) 

< e^p{-f^i(p* + erp\p*)) (14) 

< ^ <^^) 

where l\12\i follows from LemmaO ()13p is due to the fact that {z,p) is monotonically decreasing 
with respect to z G (p, !)> (|14p follows from Lemma [6l and (jlSp follows from the assumption about 
7- 

For Case (iii), applying Theorem 1 with z = {1 + er)p < ^, we have 



Pr{p > (1 + er)p} = Frl^Xi> 



nz > 

li=i J 

< ex.p{n^B{p + £rP,p)) (16) 

< exp{n^B{p* + £rP*,P*)) (17) 
= exp(n^B(p* + ea,P*)) 



where ()16p follows from Lemma [5l (jl7p follows from the first statement of Lemma [T2| and (llSp 
follows from the assumption about n. 

In summary, we have shown Pr{p > (1 + £r)p} < | for all cases. This completes the proof of 
the lemma. 

□ 

Lemma 15 Pr{p < (1 — £r)p} < | for any p € {p*, 1). 
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Proof. To prove the lemma, we shall consider the following two cases: 
Case (i): (1 - er)p > ^; 
Case (ii): (1 - er)p < ^. 

For Case (i), applying Theorem 1 with z = (I — er)p > have 

( [7/^1-1 ] ( ] r h/z] 

Pr{p < (1 - er)p} = Pr j ^ X, < 7 | = Pr j ^ X, < 7 - 1 I < Pr j ^ X, < 

< cxp(^r7/zl (19) 

< exp(7^i(z,p)) (20) 

< exp(7^i(p*-e,p*,p*)) (21) 

< cxp(7.^i(p* + erP*,P*)) (22) 

< 2' 

where ()19p follows from LemmaO ()20p is due to the fact that {z,p) is monotonically increasing 
with respect to 2; G (0,p), (j2ip follows from Lemma O and (j22p follows from Lemma [71 
For Case (ii), applying Theorem 1 with z = (1 — er)p < ^, we have 

Pr{p < (1 — £r)p} < exp {n^B{p — £rP,p)) (23) 

< exp{n^B{p* - £rP*,P*)) (24) 

< exp(n^B(p* + erP*,P*)) (25) 
= exp {n^Bip* + ea,P*)) < ^ 

where (I23p follows from Lemma [5l (|24p follows from the second statement of Lemma [T2| and (125p 
follows from Lemma [T3l 

In summary, we have shown Pr{p < (1 — £r)p} < | for both cases. The lemma is thus proved. 

□ 



Lemma 16 Pr{p > p + Sa} < | for any p € (0, p*] . 

Proof. To prove the lemma, we shall consider the following two cases: 
Case (i): p + ea>^; 
Case (ii): p + ea< ^■ 
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For Case (i) , applying Theorem 1 with z = p + ^ ^ > we have 
Pr{p>p + Sa} = Pr| ^ X, <exp('[7/zj ^ 



7 

,P 



= exp ^7 {j:^J^yP^^ - (^'^)) 

< expi^^iip* +erP\p*)) (26) 
S 

< 2' 



where ()26p follows from Lemma El 

For Case (ii) , applying Theorem 1 with z = p + < ^ , we have 



Pr{p>p + e4 = Pri^Xi> 



nz > 

< ex.p{n^B{p + £a,p)) (27) 

< exp(n^B(p'' + ea,P*)) (28) 
6 

< 2 



where (j27p follows from Lemma [5l (|28p follows from the first statement of Lemma [TUJ 

2 



In summary, we have shown Pr{p > p + Sa} < | for both cases. The lemma is thus proved. 



□ 



Lemma 17 Pr{p < p — £a} < | for any p G (0,p*]. 

Proof. To prove the lemma, we shall consider the following three cases: 
Case (i): p < Sa] 
Case (ii): p - > ^; 
Case (iii): < p - < ^• 

For Case (i), it is obvious that Pr{p <p — ea} = 0<|. 
For Case (ii), applying Theorem 1 with 

C [7/^1-1 \ \ . . XX 

^^{p<p-ea} = Prj X, <7| <Pr| X, <7| <exp(^[7/2l .^b(^P^,pJJ 

= exp(j^i( ^ -I ) ) < exp (7 (z,p)) = exp (7^i(p - 



\ilA 

< exp{j^i{p* -ea,p*)) (29) 

< exp{j^i{p' + ea,p*)) (30) 



6 

< 2 
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where (j29p follows from the second statement of Lemma [HI and (|30p follows from Lemma [9l 
For Case (iii) , applying Theorem 1 with z = p — < ^ , we have 

Pr{p<p-e4 = Pr|^Xi<nz| 

< exp{n^B{p- £a,p)) (31) 

< exp{n^B{p* -£a,P*)) (32) 

< exp{n^B{p* + £a,P*)) (33) 
S 

< 2 

where ()3ip follows from Lemma El ()32p follows from the second statement of Lemma [TOl and (I33p 
follows from Lemma [TTl 

In summary, we have shown Pr{p < p — Sa} < | for all cases. The lemma is thus proved. 

□ 

Now we are in a position to prove Theorem 3. To show Pr {|p — p| < Ea or |p — p| < SrP} > 
1 — 5, it suffices to show Pr {|p — p\ > Sa, \p — p\ > £rP} < 5 for < p < 1. 
For p G (p*, 1), we have 

Pr {|p -p\> £a, \P-P\ > £rP} = Pr{\p-p\ > ErP} = Pr{p > (1 + £r)p} + Pr{P < (1 - £r)p} 

< In p^) 

= 5 

where p4p follows from Lemmas [T^ and [T5l Similarly, for p € (0,p*], we have 

Pr {\p - p\ > Ea, \p - p\ > SrP} = Pr{\p-p\>Ea} = Pr{p>p + Ea} + Pr{p<p-Ea} 

< 2 + 2 
= 6 

where (|35p follows from Lemmas [TUl and [T71 This completes the proof of Theorem 3. 



D Proof of Theorem 4 

Lemma 18 Let I denote the support of p. Suppose the intersection between open interval {p' , p") 
and set is empty. Then, {•& ^ I : p < ^(i?)} is fixed with respect to p ^ (p' , p"). 

Proof. Let p* and p^ be two distinct real numbers included in interval (p', p"). To show the 
lemma, it suffices to show that {d ^ I : p* < J^{"d)} = {i? G / : p* < .if (t?)}. First, we shall show 
that {i? G / : < if (t?)} C {i? G / : < ^(i?)}. To this end, we let w G {?? G / : p* < ^(i?)} 
and proceed to show G {^9 G / : p* < ^(i?)}. Since vj G I and p* < ^{w), it must be true 
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that G / and < S£(w\ If this is not the case, then we have -p" > > ££(w) > p* > p' . 
Consequently, .if(tu) is included by both the interval {p', p") and the set ly. This contradicts 
the assumption of the lemma. Hence, we have shown G {i? G / : < ^(t?)} and accordingly 
{{} ^ I : p* < ^{'d)} C {i? G / : < ^(i?)}. Second, by a similar argument, we can show 
{^el:p^< ^{-d)} C {t? G / : < ^{'d)}. It follows that {i? G / : p* < = {t? G / : < 

Finally, the proof of the lemma is completed by noting that the above argument holds 
for arbitrary p* and p* included in the open interval (p', p"). 

□ 

By virtue of Theorem 1, we can show the following lemma. 

Lemma 19 Pr{p < z \ p} is monotonically decreasing with respect to p. Similarly, Pr{p > z \ p} 
is monotonically increasing with respect to p. 

Lemma 20 Let p' < p" be two consecutive distinct elements of ly H [a, 6] U {a, 6}. Then, 

limPr{p' + e < ^(p) | p' + e} = Pr{p' < ^(p) j p'}, 
limPr{p" - e < ^(p) | / - e} = Pr{/ < if(p) | 

Moreover, Pr{p < ^(j>) \ p} is monotone with respect to p & (p' ,p")- 

Proof. First, we shall show that lim^io Prjp' + e < -^{p) \ p' + ^} = Pr{p' < ^{p) \ p'}- Let 
m~^{e) be the number of elements of {^9 G / : p' < ^(-i?) < p' + e}, where / denotes the support of 
p as in Lemma [THJ We claim that Uui^^q m'^ (e) = 0. It suffices to consider two cases as follows. 

In the case of {t? G / : p' < = 0, we have m+(e) = for any e > 0. In the case of {?9 G / : 

p' < if (t?)} / 0, we have m+(e) = for < e < e*, where e* = min - p' : p' < G /} 

is positive because of the assumption that ly has no closure points in [o, b]. Hence, in both cases, 
lim^^Qm^{e) = 0. This establishes the claim. 

Noting that Pr{p' < ^{p) < p' + e | p' + e} < m^(e) as a consequence of Pr{p = i) \ p' + e} < 1 
for any t9 G /, we have that limsup^|Q Pr{p' < ^{p) < p' + e \ p' + e} < lime^o rn'^{e) = 0, which 
implies that lim^io Pr{p' < -^{p) < p' + e | p' + e} = 0. 

Since {p' + e < if(p)}n{p' < < p' + e} = and {p' < ^(p)} = {p' + e < ^(p)}U{p' < 

^(p) < p' + e}, we have Pr{p' < .if (p) | p' + e} = Pr{p' + e < ^(p) | p' + e} + Pr{p' < ^(p) < 
p' + e I p' + e}. Observing that Pr{p' < »Sf (p) | p' + e} is continuous with respect to e G (0, 1 — p'), 
we have lim^o Pr{p' < ^(p) | p' + e} = Pr{p' < ^(p) | p'}. It follows that 

lini Pr{p' + e < ^{p) \p' + e} = lim Pr{p' < ^{p) | + e} - lim Pr{p' < if (p) < p' + e | p' + e} 

= limPr{p' < ^(p) I p' + e} = Pr{p' < ^(p) | p'}. 
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Next, we shall show that hm^o Pr{p" - e < ^{p) I p" - e} = Pr{p" < ^{p) I p"}- Let m~(e) 
be the number of elements of {i9 € / : — e < < p"}. Then, we can show lim^io rn~{e) = 

by considering two cases as follows. 

In the case of {i? S / : J^{'d) < p"} = 0, we have m~{e) = for any e > 0. In the case of 
{'del: ^{'d) < p"} / 0, we have m~(e) = for < e < e*, where e* = min{p" - ^{'d) : G 
/, < p"} is positive because of the assumption that I^j// has no closure points in [a, b]. Hence, 

in both cases, lim^iQ m~ (e) = 0. It follows that lim sup^^o P'^Ij'" ~ ^ ^ ■^(p) < p" I p" ~ ^} ^ 
lim^io m~{e) = and consequently lim^io Pr{p" — e < ^(p) < p" \ p" — e} = 0. 

Since {p" - e < if (p)} = {p" < if (p)} U {p" - e < ^{p) < p"] and {p" < if (p)} n {p" - e < 
i*(p) < p"} = 0, we have Fi{p" - e < i^(p) | p" - e} = Pr{p" < if (p) | p" - e} + Pr{p" - e < 
^{p) < p" I p" - e}. 

Observing that Pr{y < if(p) | p" — e} is continuous with respect to e € (0,y), we have 
hm^oPrfr" < i^(p) | p" - e} = Pr{p" < i^(p) | p"}. It follows that lim^o Prlp" - e < if(p) I 
p" -e} = lim^olp" < ^(P) I P"}- 

Now we turn to show that Pr{p < i?(p) | p} is monotone with respect to p € {p' ,p")- 
Without loss of generality, we assume that if (.) is monotonically increasing. Since p' < p" are 
two consecutive distinct elements of fl [a,b] U {a,b}, we have that the intersection between 
open interval {p',p") and set is empty. As a result of Lemma \T8\ we can write Pr{p < if(p) | 
p} = Pr{p > I p}, where ■!? G [0, 1] is a constant independent of p G {p',p")- By Lemma [T9l we 
have that Pr{p > | p} is monotonically increasing with respect to p G {p',p")- This proves the 
monotonicity of Pr{p < if(p) | p} with respect to p G (p' ,p")- The proof of the lemma is thus 
completed. 

□ 

By a similar method as that of Lemma [20l we can show the following lemma. 

Lemma 21 Let p' < p" be two consecutive distinct elements of h// fi [a, 6] U {a, 6}. Then, 

limPr{p' + e > ^(p) I p' + e} = Pr{p' > '^(p) | p'}, 
limPr{p" - e > ^(p) I p" - e} = Pr{p" > ^(p) | p"}. 

Moreover, Pr{p > '^(p) | p} is monotone with respect to p e {p',p")- 

Now we are in a position to prove Theorem HI Let C(p) = Pr{p < if(p) | p}. By Lemma [20l 
C(p) is a monotone function of p G (p' ,p"), which implies that C(p) < max{C(p' + e), C{p" — e)} 
for any p G {p',p") and any positive e less than min{p — p', p" — p}. Consequently, 

C(p) < limmax{C7(p' + e), C{p"-e)} = max{lim C(p' + e), limC(p"-e)} < max{C7(p'), C(p")} 

eiO eiO ej,0 

for any p G {p',p")- Since the argument holds for arbitrary consecutive distinct elements of 
{if(p) e (a, 6) I p G /} U {a, 6}, we have established the statement regarding the maximum of 
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Pr{p < ^{p) I p} with respect to p E (a, By a similar method, we can prove the statement 
regarding the maximum of Pr{p > ^ (p) | p} with respect to p G (a, 6). This concludes the proof 
of Theorem m 

E Proof of Theorem 6 

The theorem can be established by showing the following lemmas. 
Lemma 22 Pr{p > p} < |. 
Proof. By Theorem 1, 



Pr{p < z} 



Pr{E.[l('^"'^. <7} for7<nz, 
Pr {E"=i -^i — "--^l 7 



Since Xj must be either or 1 and 7 is an integer, we have Pr{E[li^^ ^ -''^i < 7} < Pr{E[li^^ -'^i < 
7}. Hence, 

'Pi-{El2('^ < 7} for7<n^, 
Pr{E"=i -'^i < "^2} for 7 > riz- 
Since Xi,X2, • • • are i.i.d. Bernoulli random variables, we have Pr{p < z} < where 



Pr{p < z} < 



EZ=o (^^{^Vni -P)^^/^^-^ for ^ < z < 1, 
ESOpKI-pT-' forO<z<^. 



Let z* G [0, 1] be the largest number such that Pr{p < z*} < |. Since p is a discrete random 
variable bounded in [0,1], it must be true that Pr{p < z*} > |. Observing that ^{z,p) is 
monotonically decreasing with respect to p € (0, 1), we have 

{p>p} = {p>p,k<n}C l^^{p,p) < ^{p,p) - ^} ^ < ^ 

Noting that | < Pr{p < z*} < 'i^{z*,p) and that 'i^{z^p) is non-decreasing with respect to 

2- 



z e (0, 1), we have {p > p} C {?^(p,p) < f } ^ {^(p,p) < ^ {p < z*}. It follows that 

Pr{p > p} < Pr{p < z*} < |. 



□ 



Lemma 23 Pr{p < p} < |. 
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Proof. By Theorem 1, 



Pi{p > z} 



Since Xi,X2, ■ ■ ■ are i.i.d. Bernouhi random variables, we have Pr{p > z} = J^{z,p) where 



fz\-r forl<z<l, 
n — — ' 



for < z < ^ . 



Let z* € [0, 1] be the smahest number such that Pr{p > z*} < ^. Since p is a discrete random 
variable bounded in [0,1], it must be true that Pr{p > z*} > |. Observing that J^{z,p) is 
monotonically increasing with respect to p G (0, 1), we have 

{p < p} = {p < p, k > 0} c jjf (p,p) < Jf{p,p) = ^ !> c <{ Jf{p,p) < ^ 

Noting that | < Pr{p > z*} = .^{z*,p) and that .^{z,p) is non-increasing with respect to 
z G (0, 1), we have {p < p} C {.j^{p,p) < |} C {je{p,p) < J^{z*,p)} C {p > z*}. It follows 
that Pr{p < p} < Pr{p > z*} < |. 

□ 



F Proof of Theorem 7 

Theorem 7 can be shown by using the following result (a slight modification of Hoeffding's in- 
equality [7]) and a similar argument as that of Theorem 3. 

Lemma 24 Let Xi,--- ,Xn be random variables with joint distribution given by Then, 
Pr I ^'^^ ^' > z| < exp {n^Biz,p)) for 1 > z > p = . Similarly, Pr | ^'^^ ^' < z| < cxp (n^B(-z,p)) 
for < z < p. 

Proof. For z = p, we have Pr | ^'^^ ^' > z| < exp (n.^B(z,p)) ~ 1. For p < z < 1, it was shown 

by Hoeffding in [7j that Pr|^S^ > < exp (n.-#B(^,p))- For z = 1, Pr|^%L£i > z| = Pr{X, = 

1, * = ••,«}= O/C) <p" = cxp(n.^B(l,p)). 

For z = 0, Pr < 2} = Pr{X, = 0, z = 1, • • • , n} = < = exp (n^B(0,p)). 

For < z < p, it was shown by Hoeffding in [7] that Pr | '^'=^ ^' < < exp (ri^B(z,p)). For z = p, 

we have Pr | ^'^^ ^' < z| < exp (n^B(2,p)) = 1- 

□ 
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G Proof of Theorem 8 



By the same argument as that of Theorem 1, we can show the following lemma. 
Lemma 25 For any z > 0, 

Prjp < z\ ~ < Prjp > z\ ~ i 

\ Pr {Z]r=i <nz} for J > nz PriSr^i -'^i > "-^l /c"^ 7 > nz. 

By applying Lemma [251 we can show the following lemma. 

Lemma 26 Pr{p < z \ M} is monotonically decreasing with respect to M. Similarly, Pr{p > z \ 
M} is monotonically increasing with respect to M . 

Now we shall introduce some new functions. Let pQ < pi < ■ ■ ■ < pj he all possible values 
of p. Define random variable R such that Pr{i2 = r} = Pr{p = pr} for r = 0, 1, • ■ ■ , j. Then, 
'^(p) = '^(pr). We denote '^{pn) as U{R). Clearly, U{.) is a non-decreasing function defined 
on domain {0, 1, • • • By a linear interpolation, we can extend IA{.) as a continuous and 

non-decreasing function on [0,j]. Accordingly, we can define inverse function IA~^{.) such that 
U-^{e) = max{x G [0,i] : U{x) = 9} for ^(0) < 6 < '^{3). Then, > U{R) ^ R < 
U-^{9) ^R< g{e) where g{e) = [U-^{e)\. 

Similarly, ^(p) = ^{pr). We denote ^{pr) as C{R). Clearly, £(.) is a non-decreasing 
function defined on domain {0, 1,-- - By a linear interpolation, we can extend C{.) as a 

continuous and non-decreasing function on [0,j]. Accordingly, we can define inverse function 
C-^{.) such that C~^{e) = min{x e [0,i] : C{x) = 9} for ^(0) <e< if(j). Then, < C{R) ^ 
R > £-1(0) ^ R> h{e) where = \C~^{ey]. 

Lemma 27 Lei < r < j. Then, h{m) = r + 1 for C{r) < m < C{r + 1). 

Proof. Clearly, h{m) = r + 1 for m = C{r + 1). It remains to evaluate h{m) for m satisfying 
C{r) <m < £{r + 1). 

For m > >C(r), we have r < /^"""^(m), otherwise r > C~^{m), implying C{r) > m, since C{.) is 
non-decreasing and m ^ {'C(r) : < r < j}. For m < £(r-|-l), we have r+1 > £~^(m), otherwise 
r + l< C~^{m), implying C{r + 1) < m, since £(.) is non-decreasing and m ^ {'C(r) : < r < j}. 
Therefore, we have r < C~^{m) < r + 1 for /^(r) < m < C{r + 1). Hence, r < [/^"-"^(m)] < r -|- 1, 
i.e., r < /i(m) < r -|- 1. Since h{m) is an integer, we have h{m) = r + 1 for >C(r) < m < C{r + 1). 

□ 



Lemma 28 Let < r < j. Then, g{m) = r for U{r) < m < lA{r -\- 1) 
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Proof. Clearly, g{m) = r for m = U{r). It remains to evaluate g{m) for m satisfying U{r) < 
m < U{r + 1). 

For m > U{r), we have r < lA~^{m), otherwise r > l/(~^{m), implying h{{r) > m, since h{{.) is 
non-decreasing and m ^ {l^ir) : < r < j}. For m < h{{r+l), we have r+1 > U~^{m), otherwise 
r+l< U~^{m), implying U{r+ 1) < m, since hi{.) is non-decreasing and m ^ {l^{r) : < r < j}. 
Therefore, for 'U{r) < m <U{r + 1), we have r < U~^{m) < r + 1. Hence, r < \p{~^{m)\ < r +1, 
i.e., r < g{m) < r + 1. Since g{m) is an integer, we have g{m) = r for U{r) < m < U{r + 1). 

□ 

Noting that Pr{M > "^{p) | M} = Pr{M > U{R) \ M}, we have Pr{M > ^(p) | M} = 
Pr{ii < g{M) I M}. Let < r < j. By Lemma [281 we have that (/(m) = r for Z//(r) < m < 
U{r + 1). Observing that Pr{M > ^(p) | M} = for < M < ^(0) and that Pr{M > ^(p) | 
M} = 1 for ^ (j) < M < N, we have that the maximum of Pr{M > (p) \ M} with respect 
to M G [a, b] is achieved on IJr=o{"^ ^ ^] • ^('^) < ^(^^ + 1)} U {a, Now consider the 
range {m G [a, 6] :U{r) < m < U{r + 1)} of M. We only consider the non-trivial situation that 
U{r) < U{r + 1). For U{r) <M< U{r + 1), we have 

Pr{M >'^{p)\ M) = Pr{i? < g{M) \ M} = Vt{R < r \ M} = Pr{p < | M}, 

which is non-increasing for this range of M as can be seen from Lemma [26l By virtue of such 
monotonicity, we can characterize the maximizer of Pr{M > ^ (p) | M} with respect to M on 
the set \m G [a, : U[r) < m < iY(r -|- 1)} as follows. 
Case (i): h < U{r) or a > l/{{r + 1). This is trivial. 

Case (ii): a <lA{r) <b< U{r + 1). The maximizer must be among {U{r), b}. 
Case (iii): U{r) < a <b < lA{r + 1). The maximizer must be among {a, 6}. 
Case (iv): hl{r) < a < U{r -|- 1) < 6. The maximizer must be among {a, lA{r + 1)}. 
Case (v): a < lA{r) < lA{r -1- 1) < 6. The maximizer must be among {W(r), lA{r + 1)}. 
In summary, the maximizer must be among {lA{r), U{r -|- 1), a, 5} fl [a, 6]. It follows that the 
statement on Pr{M > (p) | M} is established. 

Next, we consider Pr{M < ^(p) | M}. Noting that Pr{M < ^(p) \ M} = Pr{M < C{R) \ 
M}, we have Pr{M < ^(p) | M} = Vi{R > h{M) \ M}. Let < r < j. By Lemma [271 we 
have that h{m) = r -|- 1 for C{r) < m < C{r + 1). Observing that Pr{Af < »Sf(p) | M} = 1 
for < M < ^{0) and that Pr{A/ < ^(p) | Af} = for if(j) < M < N, we have that the 
maximum of PrjA/ < ^(p) | Af} with respect to Af G [a, 6] is achieved on U:^ZQ{m G [a, 6] : 
C{r) < m < C{r + 1)} U {a, b}. Now consider the range {m G [a, b] : C{r) < m < C{r + 1)} of M. 
We only consider the non-trivial situation that C{r) < C{r + 1). For C{r) < M < C{r + 1), we 
have 

Pr{Af < if (p) I M} = Pr{R > h{M) \ M} = Fr{R > r + 1 \ M} = Pr{p > pr+i \ Af}, 
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which is non-decreasing for this range of M as can be seen from Lemma [26j By virtue of such 
monotonicity, we can characterize the maximizer of Pr{M < -Sf (p) | M} with respect to M on 
the set {m e [a, h\ : C{r) < m < C{r + 1)} as fohows. 
Case (i): b < C{r) or a > C{r + 1). This is trivial. 

Case (ii): a < C{r) < b < C{r + 1). The maximizer must be among {jC{r), b}. 
Case (iii): C{r) < a < b < C{r + 1). The maximizer must be among {a, b}. 
Case (iv): C{r) < a < C{r + 1) < 6. The maximizer must be among {a, £(r + 1)}. 
Case (v): a < C{r) < C{r + 1) < 6. The maximizer must be among {C{r), C{r + 1)}. 
In summary, the maximizer must be among {>C(r), C{r + 1), a, 6} fl [a, b\. It follows that the 
statement on Pr{M < ^(p) | M} is established. 
This concludes the proof of Theorem [8j 



H Proof of Theorem 10 

The theorem can be established by showing the following lemmas. 
Lemma 29 Pr{M > M„} < |. 

Proof. Since Xi must be either or 1 and 7 is an integer, we have Pr{Ell('^"'^. < 7} < 
Pi'iSlli^^ ^» ^ 7}- Hence, by Lemma [ 



Pr{Ell('^ X^ < 7} for 7 < nz, 
Pi' {^7=1 -^i — for ■y > nz 



Pr{p < z} < 
and Pr{p < z} < '^^{z,p), where 

' Eart')r„-f)/ffl foro<.<2 

with p = Let z* € [0, 1] be the largest number such that Pr{p < z*} < |. Since p is a discrete 
random variable bounded in [0, 1], it must be true that Pr{p < 2;*} > |. Observing that ^(z,^) 
is monotonically decreasing with respect to p G {j^ : i = 0,1, ■ ■ ■ , N}, we have 

where p = ^^^^ ■ Noting that | < Pr{p < z*} < ^{z*,p) and that '^{z,p) is non-decreasing 
with respect to z G (0, 1), we have {p > p} C {'^{p,p) < |} ^ {^{p,p) < '^{z*,p)} C {p < z*}. 
It follows that Fr{p >p}< Pr{p < z*} < |, which implies that Pr{M > M„} < |. 

□ 



Lemma 30 Pr{M < M/} < |. 
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Proof. By Lemma [25l we have Pr{p > z} = Jif{z,p), where 



i=7 \ i J \l-y/z]-iJ / \l-y/z]) 

with p = Let z* G [0,1] be the smahest number such that Pr{p > z*} < |. Since p is a 
discrete random variable bounded in [0, 1], it must be true that Pr{p > z*} > |. Observing that 
J^{z,p) is monotonically increasing with respect to p G {-^ : i = 0,1, ■ ■ ■ , N}, we have 

{p<p}C (^Jf{p,p) < J^{p,p) < || c < I 

where p = ■ Noting that | < Pr{p > z*} = J^{z*,p) and that J^{z,p) is non-increasing 
with respect to z G (0, 1), we have {p < p} C {Ji^{p,p) < f } C {Jlf{p,p) < Jif{z*,p)} Q {p > 



z*}. It follows that Pr{p < p} < Pr{p > z*} < |, which imphes Pr{Af < M/} < 



2- 

□ 



I Proof of Theorem 11 

In the case of M < 7, we have n = n and ^ > ^ > N, from which the theorem immediately 
follows. It remains to show the theorem for the case of M > 7. Notice that 

{n ^ n ( m—1 m 

Y,Xi<^\ + Y,mpA ^Xi<7, 
i=l ) m=l I 1=1 i=l 

{n 1 r "1 

<7l + J^nPri J^X, <7, ^Xi>-f\=n. 
i=l ) m=l I i=l i=l ) 

Since M = ^f^i Xi>^ and Xi is non-negative, we have 

N frn-1 m ~1 f " ~1 

m=n+l K i=l i=l ) I i=l ) 

Hence, 

E[n] = ^ nPri ^X,<7, ^X,>7 +E^"PM E^'<^' E^^^^ 

m— n+1 i—1 i—1 ) m—1 i—1 i—1 

N (ni-l m ^ 

< ^7«Pri^X, <7, >7UE[m], 

m—1 2—1 z— 1 J 

where m is the sample number of the classical inverse sampling scheme with the following stopping 
rule: Sampling without replacement is continued until 7 units possessing the attribute have been 
observed. By the definition of the classical inverse sampling, we have 'Yll^i -^i ~ 7- Noting that 
Xi,X2, ■ ■ ■ , X]sf are identical but dependent Bernoulli random variables with common mean p and 
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that {n > k} depends only on Xi, ■ ■ ■ , Xk-i for 1 <k < N can conclude that Wald's equation 
still applies. Hence, IE[X]i^i^«] = ]E[m] E[Xi] = 7, which implies that E[m] = ^yx-] ~ p' Since 
E[n] is less than both n and E[m] as shown above, we have E[n] < min{n, This completes the 
proof of the theorem. 



J Proof of Theorem 12 

The theorem can be established by showing the following lemmas. 
Lemma 31 Pr{A > A} < |. 

Proof. By Theorem 1, we have 

Pr{Ell('^ X^ < 7} for 7 < nz, 
Pr {Z]"=i ^ nz} for 7 > nz. 

and thus Pr{A < z} = '^{z, A), where 

EZ=o l^.m^ - l)A]^exp(-(rfl - 1)A) for z > I 



Pr{A < z} = 



E1=o I! (™A)^ exp(-nA) for < z < ^. 



Let > be the largest number such that Pr{A < z*} < |. Since A is a non- negative discrete 

s 

2- 



random variable, it must be true that Pr{A < z*} > ^. Observing that 'i^{z, A) is monotonically 



decreasing with respect to A G (0, oo), we have 



{A > A} C ^ ^^(A, A) < ^(A, A) = ^1 C |^^(A, A) < ^ 



Noting that ^ < Pr{A < z*} = '^{z*,\) and that '^{z,\) is non-decreasing with respect to 

2^ 



z G (0,c3o), we have {A > A} C {^(A, A) < |} C {^(A,A) < ^(z*,A)} C {A < z*]. It follows 
that Pr{A > A} < Pr{A < z*} < |. 



□ 



Lemma 32 Pr{A < A} < |. 



Proof. By Theorem 1, we have 

'Pi-{Ellf^^. >7} for7<nz, 
P'^iSiLi — nz} for J > nz 



Pr{A >z} = 
and thus Pr{A > z} = J^{z, A) where 
J^{z,X) -- 



Zr=,m\Xyexp{-ll\X) forz>I, 



E£r-lirWexp(-nA) forO<z<^. 
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Let z* > be the smallest number such that Pr{A > z*} < |. Since A is a non-negative discrete 
random variable, it must be true that Pr{A > z*} > |. Observing that Jif{z, A) is monotonically 
increasing with respect to A G (0, oo), we have 

{A < A} = {A < A, k > 0} C |^^(A, A) < ^^(A, A) = ^ ^ |^^(A, A) < ^| . 

Noting that | < Pr{A > z*} = Jif{z*,X) and that J^{z,X) is non- increasing with respect to 
z e (0, oo), we have {A < A} C {J^(X, A) < |} C {J^(X, A) < je{z*,X)} C {A > z*}. It follows 
that Pr{A < A} < Pr{A > z*} < |. 

□ 



K Proof of Theorem 13 

By the same method as that of Lemma [TH we have 
Lemma 33 Pr{^ > (1 + ^r)/"} < | for any fj. G {p* , 1). 

Lemma 34 Let < < 1, z = {I - er)f^ and e' = 1 - . Suppose 7 > Then, 

^1 < ^l{il-e')^i,n) for any p. G (0,1). 

Proof. As a consequence of 7 > we have < < fi for any fi G (0, 1). Since ^i{w, fi) is 

monotonically increasing with respect to w; G (0, /x), it suffices to show that < {l — e')fi for any 
fie (0,1). That is, to show < il-e')fi, G (0, 1), i.e., < 1 - e', V/iG(0,l). 

This follows from the definition of e'. □ 



Lemma 35 Pr{/i < (1 — erO^} < | for any /i G {p*, 1). 

Proof. To prove the lemma, we shall consider the following two cases: 
Case (i): (1 - er)n > ^; 
Case (ii): (1 - e^)^ < ^• 

For Case (i), applying Theorem 1 with z = (1 — e,.)/i > ^, we have Pr{/i < (1 — e,-)yu} = 
jX^ili^^""^ ATj < 7|. As a consequence of 7 > we have < p^yj -j ,;^ < < /i for any 

fi G (0, 1). Hence, applying Lemma [5l we have 

Pr{/i < (1 - e,)^} < cxp((r7/z] -1).^B (^T^ 



7/^1 - 1 ' 



exp I 7 I J — ) ) < exp ( 7 ( — , /.J 
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By Lemmas 1341 and [6l 

Pr{/i < (1 - er)fi} < exp (7^i((l - e')fi,fi)) < exp (7^i((l - e')p*,p*)) 

For Case (ii), applying Theorem 1 with z = {1 — er)fJ' < and by a similar argument as that 
of Lemma [T5l we have Pr{/x < (1 — £r)^^} < |- 

In summary, we have shown Pr{/i <(1 — er)^}<| for all cases. The lemma is thus proved. 

□ 

By the same method as that of Lemma [T6l we have 
Lemma 36 Pr{/x > fJ, + Ea} < | for any fi € (0,p*]. 

Lemma 37 Let z = fi — e. Suppose < < /x. Then, ^\ (^T^,fJ^ is monotonically increasing 
with respect to ii ^ [e, ^) . 



Proof. Note that = i _ J, in (i^^) ^ - (1 - 1) ^, where = ^ = -7 + ^ and 

t = = Hence, 

1 f u — w\ a — w 

' ' > 



- ^l) since :^ - ^ ^ 



if z2(l-u;) < l^{l-^t)w, i.e., z2 (i - 1) < ^ z2 - i - 1) < ^ ;^(i_;2)_£i < 

7' 



□ 



Lemma 38 Pr{^ < ^ — ^a} < f /o?" any /x G (0,j5*]. 

Proof. To prove the lemma, we shall consider the following three cases: 
Case (i): n < Sa', 
Case (h): /x - > ^; 
Case (iii): < /i - < ^. 



For Case (i), it is evident that Pr{/i </i — ea} = 0<|. 



2- 
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For Case (ii), applying Theorem 1 with z = /i — > ^, we have Pr{/1 < /i - Eq} = 
Pr < 7|. From the definition of z and the assumption that 7 > we see that 

< |-^/2]„]^ < < /U for any fj, € (0,p*]. Hence, it follows from Lemma [5] that 

PT{p.<H-ea} < exp ^(f7/z] - 1) 

= exp ^7 ^ ^^^J^ < exp ^7 
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Invoking Lemma [371 we have 



7 + er - 1 // 2 

where e' is defined in Lemma [34l 

For Case (iii), applying Theorem 1 with z = fi — Sa < ^ and by an argument as that of Lemma 
[T71 we have Pr{/x < /i — £0} < |- In summary, we have shown Pr{/i < ^ — Ca} < | for ah cases. 
The lemma is thus proved. 

□ 



Finally, the proof of Theorem 13 can be accomplished by a similar argument as that of Theorem 

3. 

L Proof of Theorem 14 

The theorem can be established by showing the following lemmas. 



Lemma 39 Fv{fi >JZ}<^ 
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Proof. For /i > z > ^, by Theorem 1 and Lemma [U we have 

( [7/^1-1 ] . 

Pr{/i<0} = Pr< ^ X, <7i <cxp(^([7/2l-l)^B 
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[j/z] - 1 



exp ^7^1 ^j-y-y— j < exp ^7.^1 ^;:y^,A* 

where the last inequality is due to < and the fact that ^i{z,fi) is monotonically 

increasing with respect to z G (0,/i). For fi > z and < z < ^, by Theorem 1 and Lemma [5l we 
have Pr{/i < z} ~ Pr | ^' < zj < exp(n.y#B(-z, m))- Therefore, Pr{^ < z} < ^{z,fi), where 

exp (^7^1 (^:^,^^^ for ^ < z < /i, 
exp(n^B(-2, /^)) for < z < ^, z < fj,. 



^{z,^i) 



Let z* € [0, 1] be the largest number such that Pr{/i < 2*} < |. Then, it must be true that either 
Pr{/x < z*} > I or Pr{^ < z*} = |. Observing that is monotonically decreasing with 

respect to /i S (z, 1), we have 

{M>7t} = {/i>P>/i, k<n}C |^(/i,/i)<^(/i,7I) = ^, Ai>P>/i| C |^(/i,;u)<^, /i<M 

In the case of Pr{^ < z*} > |, we have | < Pr{/i < z*} < '^{z* ,11). Since is increasing 

with respect to z e (0, /x), we have {/x > /!} C {'^{fi, /x) < |, /i < /x} C {^(/I, /z) < "^{z* , fi), p, < 
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lj,}C{fl< z*}. It follows that > /I} < Pr{/2 < z*} < |. In the case of Pr{/i < z*} = 2, we 
have I = Fr{p, < z*} < ^{z*,n). Since "^{zjfi) is increasing with respect to z G (0, ;u), we have 
{Ai > 71} ^ {5^(Ai,Ai) < |, A < /"} ^ {^^(m,/^) < fi < fi} ^ {fi < z*}. It follows that 

Pr{/x > 71} < Pr{/z < z*} = |. 

□ 



Lemma 40 Fr{fi < /i} < |. 

Proof. For z > fi and 1 > z > ^, by Theorem 1 and Lemma [U we have 
Pr{/i > z} = Pr < ^ X, > 7 ^ < exp [7/zJ .^b 



L7/^J 



= exp |^7^i (^"[^yT^'^j j - exp(7^i(2;,^)) 

where the last inequality is due to p^^^ ^ z and the fact that ^i{z, //) is monotonically decreasing 
with respect to z S {fi, 1). For ^ < 2; < ^, by Theorem 1 and Lemma O we have Pr{/i > z} = 
Pr I ^'=^ > z| < exp(n^B(2, m))- Therefore, Pr{/i > z} < J^{z,fj,), where 



exp (7^1 (z, /i)) for 1 > z > ^, z > //, 
ex.p{n^B{z , fJ.)) for /i < z < ^. 



Let z* G [0, 1] be the smallest number such that Pr{/x > z*} < |. Then, it must be true that 
either Pr{/i > z*} > | or Pr{/x > z*} = |. Observing that J^(z,/i) is monotonically increasing 
with respect to fi G (0, z), we have 

{yu < /£} = {m< /£</i, k> 0} C < ^(/i,/£) = M</£</il>C <j^^(/l,M) < ^, M > A* 

In the case of Pr{/i > z*} > |, we have f < Pr{/i > z*} < =^(z*,/i). Since =^(z,/i) is 
decreasing with respect to z € (/U, 1), we have {// < Ai} ^ < |, /i > /u} C < 

jr(z*,;u), > ^ {a^ > -z*}- It follows that Pr{/i < /x} < Pr{^ > z*} < |. In the case of 
Pr{p, > z*} = |, we have | = Pr{/i > z*} < J^{z*,fj,). Since J^{z,fM) is decreasing with respect 
to z G (/^, 1), we have {/U < /i} ^ {,^(jl,n) < |, /i > /i} C {J^(/x, //) < J^(z*,ax), /i > /i} ^ 
{/i > z*}. It follows that Pr{^ < W < Pr{/x > z*} = f . 

□ 
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